Shutter release using secondary camera

ABSTRACT

A method of capturing an image includes activating a first image sensor and capturing a sequence of images with a second image sensor. A determination is made as to whether the sequence of images captured by the second image sensor includes a shutter gesture. If a shutter gesture is included in the sequence of images captured by the second image sensor, the first image sensor captures a target image in response.

TECHNICAL FIELD

This disclosure relates generally to optics, and in particular but notexclusively, relates to digital imaging.

BACKGROUND INFORMATION

In many photography scenarios, it is advantageous to keep a camera asstill as possible. Without a tripod or other support, a photographer'sown movement can compromise the image quality of the photograph. Evenpressing a physical button or software generated virtual button (such ason a touch screen interface) may move the camera enough to negativelyimpact the photograph. This problem can be referred to as hand shakingor camera shake. Camera shake can be problematic with dedicated camerasand with mobile devices (e.g. tablets and smartphones) that have anon-board camera.

FIG. 1 illustrates a mobile device 103 that has an image capture button155 for initiating an image capture of scene 125 using a digital imagesensor (not shown) on the backside of mobile device 103. In FIG. 1, thedigital image sensor on the backside of mobile device 103 is activatedand is transmitting its field of view as scene 125 on the display of themobile device. Circuitry within mobile device 103 is displaying imagecapture button 150 as a virtual button to a user as an alternative to aphysical button or switch, such as image capture button 155. The usermay press image capture button 150 in order to initiate an image captureof scene 125. Regardless of capture button 155, the user's touch frompressing the image capture button may move the camera slightly, whichmay cause unwanted effects (e.g. blurring) in the captured image.

Many solutions have been proposed and marketed to address this problem.In one solution, the camera shake caused by a user pressing an imagecapture button is mitigated by delaying the image capture until afterthe initial impact of the user pressing image capture button 150 or 155.However, delaying the image capture may cause the user to missphotographing the desired scene or subject, especially in an actionshot. Another solution to the camera shake problem is to have a separatedevice (e.g. remote control) send a wireless/remote shutter signal tothe camera. The remote shutter signal allows a camera to initiate animage capture without being physically impacted. Among the drawbacks ofthis solution are the added expense of the additional hardware of theremote control, the inconvenience of controlling two devices together atthe same time, and the impracticability of taking the camera out of theuser's hand.

Some devices mitigate the camera shake problem with an anti-shakealgorithm that computes the motion estimation between two or moreconsecutive frames and compensates for the motions between the frames.One disadvantage of this solution is the need for a buffer to store theframes needed to compensate for the motion between the frames. Anotherdisadvantage of this solution is the extra processing resources andcorresponding electrical power that may be used in processing. One finalpotential disadvantage of this solution is that the image quality of theimage created by the anti-shake algorithm may be degraded if the motionbetween frames is estimated inaccurately. Given the drawbacks of theconventional solutions, it would be advantageous to have a solution thatdid not utilize an image capture time delay, did not include substantialadditional hardware and did not require intensive processing andsubstantial power resources.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the invention aredescribed with reference to the following figures, wherein likereference numerals refer to like parts throughout the various viewsunless otherwise specified.

FIG. 1 illustrates a user interacting with a mobile device with imagecapture buttons for initiating an image capture by a camera of themobile device.

FIG. 2 illustrates a user gesturing in the field of view of a motionsense image sensor of a mobile device to initiate an image capture witha primary image sensor of the mobile device, in accordance with theteachings of the present invention.

FIG. 3 illustrates a three by three matrix of sectors of an examplemotion sense image sensor and corresponding shutter vectors, inaccordance with the teachings of the present invention.

FIG. 4 is a block diagram that illustrates an example image sensorsystem that includes an example motion sense image sensor and a primaryimage sensor, in accordance with the teachings of the present invention.

FIG. 5 is a flow chart illustrating an example process of initiating animage capture that includes imaging a shutter gesture from a user, inaccordance with the teachings of the present invention.

DETAILED DESCRIPTION

Examples of a system and method for initiating an image capture with afirst image sensor using an image or images from a second image sensorare described herein. In the following description, numerous specificdetails are set forth to provide a thorough understanding of theexamples. One skilled in the relevant art will recognize, however, thatthe techniques described herein can be practiced without one or more ofthe specific details, or with other methods, components, materials, etc.In other instances, well-known structures, materials, or operations arenot shown or described in detail to avoid obscuring certain aspects.

Reference throughout this specification to “one example” or “an example”means that a particular feature, structure, or characteristic describedin connection with the example is included in at least one example ofthe present invention. Thus, the appearances of the phrases “in oneexample” or “in an example” in various places throughout thisspecification are not necessarily all referring to the same example.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more examples. In thisdisclosure, it shall be appreciated that the term “shutter” may be inreference to an electronic shutter (e.g. rolling or global shutter ofelectronic pixels), a mechanical shutter, or both an electronic shutterand a mechanical shutter used in combination to capture image light.

FIG. 2 illustrates a user holding mobile device 203, in accordance withthe teachings of the present invention. Mobile device 203 includesmotion sense image sensor 205 and a primary image sensor 250. Asdiscussed above, a conventional image capture includes pushing aphysical button/switch or pressing a virtual button presented to a userby a display with a touch screen. Of course, the conventional methodsinclude touching the camera, which may create the problem of camerashake that negatively affects the images being captured.

In FIG. 2, the user holds mobile device 203 in order to capture an imageof scene 225 with primary image sensor 250. Instead of physicallycontacting mobile device 203 to initiate an image capture with primaryimage sensor 250, the user may signal an image capture pattern toinitiate the image capture. In one example, motion sense image sensor205 captures a sequence of images (two or more images) to determine ifthe user is conveying a shutter gesture 275 to initiate an imagecapture. If the sequence of images contains shutter gesture 275, thenprimary image sensor 250 captures an image. In one example, the imagecapture pattern can be a hand signal such as holding up a particularnumber of fingers in the field of view of motion sense image sensor 205.

Many mobile devices such as tablets and smartphones contain a highresolution (e.g. 8 mega pixels) rear facing camera that is favored fortaking photographs. Many mobile devices also have a front facing camerathat sometimes has a lower pixel resolution (e.g. 1.3 mega pixels) thanthe rear facing camera. The front facing camera is often mounted facingthe opposite direction as the rear facing camera and the same directionas a display of the mobile device (e.g. liquid crystal display oractive-matrix organic light emitting diode) to enable videoteleconferencing or similar applications. However, the disclosure is notlimited to a device that contains only two image sensors, nor is itlimited to a device that has two image sensors facing substantiallyopposite directions.

In the illustrated example of FIG. 2, it may be advantageous to haveprimary image sensor 250 be a rear facing higher resolution image sensorand to have motion sense image sensor 205 be a lower resolution. Havinga lower resolution image sensor as the image sensor that images an imagecapture pattern such as shutter gesture 275 may save power and a higherresolution image sensor may not be necessary to effectively sense auser's shutter gesture. Additionally, it may be advantageous to havemotion sense image sensor 205 be on the same side of mobile device 203as a display of the mobile device so the display can present the userwith scene 225, which is in the field of view of primary image sensor250.

FIG. 3 illustrates a three by three matrix of sectors of an examplemotion sense image sensor and corresponding shutter vectors, inaccordance with the teachings of the present invention. In theillustrated example, motion sense image sensor 305 includes nine sectorsor zones and the nine sectors are arranged in a three by three matrix.However, motion sense image sensor 305 may include sectors arranged in atwo by two matrix, a four by four matrix, or even larger matrices.Additionally, the sectors may be arranged in a two by three matrix, orotherwise. Furthermore, the arrangement of the sectors may be rearrangedor reconfigured by firmware that controls motions sense image sensor305. It is appreciated that the sectors may simply be a firmware'spartitioning of the pixels in motion sense image sensor 305 and that thesectors can be reconfigured in firmware without physically alteringmotion sense image sensor 305. Motion sense image sensor 305 may includered, green, and blue filters to generate color images or motion senseimage sensor 305 may generate gray-scale images.

FIG. 3 also includes shutter vectors 310 that correspond with the threeby three matrix that corresponds with motion sense image sensor 305. Itis appreciated that motion sense image sensor 305 may be an example ofmotion sense image sensor 205. When motion sense image sensor 305captures a sequence of images to be analyzed for a shutter gesture (e.g.shutter gesture 275), the sequence may be analyzed according to shuttervectors 310. For example, if the sequence of images includes threeimages and the images show the user's hand in sector 4 in the firstimage, sector 5 in the second image, and sector 6 in the third image,then the user's hand has gestured to match vector 456. Similarly, if theuser's hand is in sector 1 in the first image, sector 5 in the secondimage, and sector 9 in the third image, then the user's hand hasgestured to match vector 159.

In one example, the shutter gesture can be programmed into mobile device203 by the user. For example, a right handed user may want the shuttergesture to be vector 357, while a left handed user may want the shuttergesture to be vector 159. In addition, different gestures may indicatedifferent shutter commands. For example, one gesture may generate ashutter signal that captures a single image with primary image sensor250, while a different gesture may generate a series of shutter signalsthat capture a burst of images (e.g. five images within a predefinedtime period). In one example, one shutter signal causes primary imagesensor 250 to capture multiple images. In one example, a shutter gesturemay generate a shutter signal that includes a corresponding exposureperiod. In one example, a first shutter gesture or image capture patterngenerates a shutter signal that corresponds with a 10 ms pixel exposuretime and a second shutter gesture or image capture pattern correspondswith a 100 ms pixel exposure time.

It is appreciated that the illustrated example in FIG. 3 is just anexample of how gestures could initiate an image capture by a camera.Other examples are possible including gesturing the letter “Z” (vector1235789), the letter “N” (vector 7415963), or otherwise. Additionally,motion sense image sensor 305 may image objects or body zones (e.g. eyesor hands) of the user and pass an image or images to control logic todetermine if a shutter gesture or image capture pattern is present inthe image or images. As briefly discussed above, an image capturepattern such as a hand signal (e.g. “thumbs up” or “peace sign”) may beused to initiate an image capture and motion sense image sensor 205 mayonly need to capture one image (as opposed to a sequence of images) toinitiate an image capture with primary image sensor 250.

FIG. 4 illustrates an example image sensor system 400 that includesmotion sense image sensor 205 and primary image sensor 250, inaccordance with the teachings of the present invention. In theillustrated example, motion sense image sensor 205 includes motion sensecontrol circuitry 420 and motion sense control circuitry 420 is coupledto transmit shutter signal 445 to primary image sensor 250. Theillustrated motion sense image sensor 205 also includes pixel array 413and readout circuitry 410, although other configurations are possible.

Pixel array 413 is a two-dimensional array of image sensors or pixels(e.g., pixels P1, P2 . . . , Pn). In one example, each pixel is acomplementary metal-oxide-semiconductor (“CMOS”) image sensor. In oneexample, pixel array 413 includes a color filter array including a colorpattern (e.g., Bayer pattern or mosaic) of red, green, and blue filters.The color filter array may be disposed over pixel array 413. Asillustrated, each pixel is arranged into a row (e.g., rows R1 to Ry) anda column (e.g., column C1 to Cx) to acquire image data of a person,place, or object, which can then be used to render a two dimensionalimage of the person, place, or object.

After each pixel has acquired its image data or image charge, the imagedata is readout by readout circuitry 410. Readout circuitry 410 mayinclude amplification circuitry, analog-to-digital (“ADC”) conversioncircuitry, or otherwise. In one example, readout circuitry 410 mayreadout a row of image data at a time along readout column lines(illustrated) or may readout the image data using a variety of othertechniques (not illustrated), such as a column/row readout, a serialreadout, or a full parallel readout of all pixels simultaneously.

In FIG. 4, readout circuitry 410 is coupled to motion sense controlcircuitry 420. Motion sense control circuitry 420 is coupled to pixelarray 413 to control operational characteristic of pixel array 413. Forexample, motion sense control circuitry 420 may generate a shuttersignal or shutter signals for controlling image or video acquisition. Inone example, the shutter signal is a global shutter signal forsimultaneously enabling all selected pixels within pixel array 413 tosimultaneously capture their respective image data during a singleacquisition window. In one example, the shutter signal is a rollingshutter signal whereby each row, column, or group of pixels issequentially enabled during consecutive acquisition windows.

Motion sense control circuitry 420 may be different than a regularcontrol circuitry (not shown) that controls pixel array 413 during anormal image acquisition or imaging mode. For example, motion sensecontrol circuitry 420 may include circuitry configured to generate ashutter signal for only a portion of the pixels in pixel array 413,rather than generating a shutter signal for all or substantially all ofthe pixels in pixel array 413, as in a regular image acquisition mode.Generating a shutter signal for only a portion of the pixels in pixelarray 413 may save power and may enable image acquisition at a highframe rate. Enabling image acquisition at a higher frame rate may alsobe advantageous to minimize a delay between recognizing a user gestureimaged by motion sense image sensor 205 and initiating an image captureby primary image sensor 250. It is appreciated that motion sense controlcircuitry 420 may share some electrical components with a regularcontrol circuitry (not shown).

In one example, motion sense control circuitry 420 generates a shuttersignal for only a portion of the pixels in pixel array 413 and thencontrols readout circuitry 410 to read out only the pixels that receiveda shutter signal. Reading out only a portion of the pixels of pixelarray 413 may save time and allow for a greater frame rate (e.g. greaterthan 30 frames per second). In one example, only a portion of thecolumns or rows are read out from pixel array 413. Reading out only aportion of the pixels is one potential way of saving power.

Motion sense control circuitry 420 may include a memory for storingimages and instructions and a processor for executing instructions. Theprocessor and memory may be shared with a regular control circuitry ofmotion sense image sensor 205. The processor or other logic withinmotion sense control circuitry 420 is capable of analyzing image datareceived from the readout circuitry and analyzing the image data anddetermining whether a shutter gesture or image capture pattern ispresent in the image data. If a shutter gesture or image capture patternis present in the image data, the processor or other control logic maytransmit shutter signal 445 to primary image sensor 250. Shutter signal445 may be an analog signal or a digital signal that includes data (e.g.exposure time) pertaining to capturing a target image.

FIG. 5 is a flow chart illustrating an example process of initiating animage capture that includes imaging a shutter gesture from a user, inaccordance with the teachings of the present invention. The order inwhich some or all of the process blocks appear in process 500 should notbe deemed limiting. Rather, one of ordinary skill in the art having thebenefit of the present disclosure will understand that some of theprocess blocks may be executed in a variety of orders not illustrated,or even in parallel.

In process block 505, a first image sensor (e.g. primary image sensor250) is activated. A sequence of images (at least two) is captured witha second image sensor (e.g. motion sense image sensor 205), in processblock 510. The sequence of images may be captured in a motion sense modethat is different than a regular image capture mode of the second imagesensor. The motion sense mode may be an ultra-low power mode thatutilizes motion sense control circuitry (e.g. motion sense controlcircuitry 420) that is different, or partially different from thecircuitry that is regularly used to capture images. In process block515, the sequence of images captured by the second image sensor areanalyzed to determine if a shutter gesture was included in the sequenceof images. A processor or other logic (e.g. FPGA) determines whether ashutter gesture was included in the sequence of images. In one example,the processor or other logic is included in motion sense controlcircuitry 420. In one example, a processor analyzes one image capturedby the second image sensor for an image capture pattern.

In process block 520, a target image is captured with the first imagesensor in response to determining the sequence of images includes ashutter gesture from a user. In the example image sensor system 400,motion sense control circuitry 420 may determine that a shutter gesturewas included in the sequences of images and transmit shutter signal 445to primary image sensor 250. Primary image sensor 250 then captures atarget image when it receives shutter signal 445. Shutter signal 445 maycontrol or include an exposure time of the first image sensor to capturethe target image and the exposure time may vary based on a category ofthe image capture pattern from the user. For example some shuttergestures or image capture patterns (e.g. hand signals) may cause primaryimage sensor 250 to capture images with different exposure times.

In process block 525, process 500 returns to process block 510 if thefirst image sensor is still activated so that the second image sensorcan keep capturing images to analyze for a shutter gesture. If the firstimage sensor is not still activated (e.g. the user has exited a cameramode or entered another application) the second image sensor isdeactivated (process block 530) and it no longer captures images toanalyze for shutter gestures.

The processes explained above are described in terms of computersoftware and hardware. The techniques described may constitutemachine-executable instructions embodied within a tangible ornon-transitory machine (e.g., computer) readable storage medium, thatwhen executed by a machine will cause the machine to perform theoperations described. Additionally, the processes may be embodied withinhardware, such as an application specific integrated circuit (“ASIC”) orotherwise.

A tangible non-transitory machine-readable storage medium includes anymechanism that provides (i.e., stores) information in a form accessibleby a machine (e.g., a computer, network device, personal digitalassistant, manufacturing tool, any device with a set of one or moreprocessors, etc.). For example, a machine-readable storage mediumincludes recordable/non-recordable media (e.g., read only memory (ROM),random access memory (RAM), magnetic disk storage media, optical storagemedia, flash memory devices, etc.).

The above description of illustrated examples of the invention,including what is described in the Abstract, is not intended to beexhaustive or to limit the invention to the precise forms disclosed.While specific embodiments of, and examples for, the invention aredescribed herein for illustrative purposes, various modifications arepossible within the scope of the invention, as those skilled in therelevant art will recognize.

These modifications can be made to the invention in light of the abovedetailed description. The terms used in the following claims should notbe construed to limit the invention to the specific embodimentsdisclosed in the specification. Rather, the scope of the invention is tobe determined entirely by the following claims, which are to beconstrued in accordance with established doctrines of claiminterpretation.

What is claimed is:
 1. A method of capturing an image, the methodcomprising: activating a first image sensor; capturing a sequence ofimages with a second image sensor; determining if the sequence of imagescaptured with the second image sensor includes a shutter gesture from auser; and capturing a target image with the first image sensor inresponse to determining the sequence of images captured with the secondimage sensor includes the shutter gesture from the user.
 2. The methodof claim 1, wherein capturing the sequence of images with the secondimage sensor is facilitated by a motion-sense mode that is differentfrom a regular image capture mode of the second image sensor.
 3. Themethod of claim 2, wherein the motion-sense mode includes acquiringpixel data from different sectors of the second image sensor to formeach image in the sequence of images, and wherein determining if thesequence of images captured with the second image sensor includes theshutter gesture includes analyzing the sequence of images for userinitiated movement in relation to the sectors.
 4. The method of claim 3,wherein only a portion of pixels available in each of the sectors areused to form each of the images in the sequence of images.
 5. The methodof claim 3, wherein the sectors are arranged in a three by three matrix.6. The method of claim 3, wherein the motion-sense mode includescapturing the sequence of images at a frame rate greater than 30 framesper second.
 7. The method of claim 1, wherein the second image sensor isincluded in a front facing image sensor on a mobile device that faces afirst direction that is approximately opposite to a second directionthat the first image sensor faces.
 8. The method of claim 7, wherein thefirst image sensor has a higher pixel resolution than the second imagesensor.
 9. The method of claim 1 further comprising displaying a fieldof view of the first image sensor to the user after activating the firstimage sensor.
 10. The method of claim 1, wherein capturing the targetimage with the first image sensor is in response to the first imagesensor receiving a shutter signal from motion-sense control circuitry ofthe second image sensor, and wherein the motion-sense control circuitrydetermines if the sequence of images includes the shutter gesture.
 11. Anon-transitory machine-accessible storage medium that providesinstructions that, when executed by a machine, will cause the machine toperform operations comprising: activating a first image sensor;capturing a sequence of images with a second image sensor; determiningif the sequence of images captured with the second image sensor includesa shutter gesture from a user; and capturing a target image with thefirst image sensor in response to determining the sequence of imagescaptured with the second image sensor includes the shutter gesture fromthe user.
 12. The non-transitory machine-accessible storage medium ofclaim 11, wherein capturing the sequence of images with the second imagesensor is facilitated by a motion-sense mode that is different from aregular image capture mode of the second image sensor.
 13. Thenon-transitory machine-accessible storage medium of claim 12, whereinthe motion-sense mode includes acquiring pixel data from differentsectors of the second image sensor to form each image in the sequence ofimages, and wherein determining if the sequence of images captured withthe second image sensor includes the shutter gesture includes analyzingthe sequence of images for user initiated movement in relation to thesectors.
 14. The non-transitory machine-accessible storage medium ofclaim 13, wherein only a portion of pixels available in each of thesectors are used to form each of the images in the sequence of images.15. The non-transitory machine-accessible storage medium of claim 13,wherein the motion-sense mode includes capturing the sequence of imagesat a frame rate greater than 30 frames per second.
 16. Thenon-transitory machine-accessible storage medium of claim 11, whereinthe second image sensor is included in a front facing image sensor on amobile device that faces a first direction that is approximatelyopposite to a second direction that the first image sensor faces. 17.The non-transitory machine-accessible storage medium of claim 11,wherein capturing the target image with the first image sensor is inresponse to the first image sensor receiving a shutter signal frommotion-sense control circuitry of the second image sensor, and whereinthe motion-sense control circuitry determines if the sequence of imagesincludes the shutter gesture.
 18. An imaging system comprising: a firstimage sensor; a second image sensor; and control circuitry coupled totransmit a shutter signal to the first image sensor and coupled toreceive image data from the second image sensor, wherein the controlcircuitry includes a non-transitory machine-accessible storage mediumthat provides instructions that, when executed by the imaging system,will cause the imaging system to perform operations comprising:capturing a sequence of images with the second image sensor; determiningif the sequence of images captured with the second image sensor includesan image capture pattern from a user; sending the shutter signal fromthe control circuitry to the first image sensor when the image capturepattern is included in the sequence of images captured with the secondimage sensor; and capturing a target image with the first image sensorin response to receiving the shutter signal from the control circuitry.19. The imaging system of claim 18, wherein capturing the sequence ofimages includes acquiring pixel data from different sectors of thesecond image sensor to create each image in the sequence of images, andwherein determining if the sequence of images captured with the secondimage sensor includes the image capture pattern includes analyzing thesequence of images for user initiated movement in relation to thesectors.
 20. The imaging system of claim 19, wherein only a portion ofpixels available in each of the sectors are used to create each of theimages in the sequence of images.
 21. The imaging system of claim 18,wherein the second image sensor is a front facing image sensor on amobile device that faces a first direction that is approximatelyopposite to a second direction that the first image sensor faces. 22.The imaging system of claim 18, wherein the image capture pattern is ahand symbol formed by a hand of the user, and wherein the image capturepattern can be determined with one image among the sequence of images.23. The imaging system of claim 18, wherein the shutter signal controlsan exposure time of the first image sensor to capture the target imageand the exposure time varies based on a category of the image capturepattern.
 24. The imaging system of claim 18 including furtherinstructions stored in the non-transitory machine-accessible storagemedium, that when executed by the imaging system, will cause the imagingsystem to perform operations comprising: capturing a second target imagewith the first image sensor in response to receiving the shutter signalfrom the control circuitry.